Native sim support#10621
Conversation
lgirdwood
requested review from
abonislawski,
dbaluta,
iuliana-prodan,
kv2019i,
lbetlej,
lyakh,
mmaka1 and
plbossart
as code owners
There was a problem hiding this comment.
Pull request overview
Adds Zephyr native_sim support to enable running SOF Zephyr tests natively (including under Valgrind) and refines POSIX/libFuzzer integration so fuzz-specific code is only built/used when enabled.
Changes:
- Add a
native_simplatform target to the Zephyr build helper. - Make POSIX fuzzing sources/IPC hooks conditional on
CONFIG_ARCH_POSIX_LIBFUZZER. - Extend the run scripts to support
native_simexecution and optional--valgrind.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
zephyr/CMakeLists.txt |
Separates POSIX sources from fuzz-only sources so fuzz.c only builds when libFuzzer is enabled. |
src/platform/posix/ipc.c |
Gates fuzz ISR/IRQ plumbing behind CONFIG_ARCH_POSIX_LIBFUZZER. |
scripts/xtensa-build-zephyr.py |
Adds native_sim as a supported Zephyr platform config. |
scripts/sof-qemu-run.sh |
Adds --valgrind flag parsing and changes default build dir behavior for native_sim runs. |
scripts/sof-qemu-run.py |
Detects native_sim from CMakeCache.txt, supports running under Valgrind, and skips QEMU monitor steps for native_sim. |
app/boards/native_sim.conf |
Introduces a board-specific Kconfig fragment for native_sim. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| #ifdef CONFIG_ARCH_POSIX_LIBFUZZER | ||
| // Not an ISR, called from the native_posix fuzz interrupt. Left | ||
| // alone for general hygiene. This is how a IPC interrupt would look | ||
| // if we had one. |
There was a problem hiding this comment.
Inside the #ifdef CONFIG_ARCH_POSIX_LIBFUZZER block, the extern declaration for posix_fuzz_buf/posix_fuzz_sz (currently extern uint8_t *posix_fuzz_buf, posix_fuzz_sz;) does not match the definitions in src/platform/posix/fuzz.c (const uint8_t *posix_fuzz_buf; size_t posix_fuzz_sz;). This mismatch can cause incorrect reads/writes (e.g., posix_fuzz_sz = 0; only updating 1 byte). Please split these into separate externs with the correct types (and const).
There was a problem hiding this comment.
under investigation why we dont use header to align.
lgirdwood
force-pushed
the
native_sim
branch
2 times, most recently
from
11e03f6 to
db6ddb6
Compare
|
|
||
| if(CONFIG_SOF_BOOT_TEST_STANDALONE AND CONFIG_ZTEST) | ||
| set(MATH_ZTEST_SOURCES | ||
| ../../test/ztest/unit/math/basic/arithmetic/test_crc32_ztest.c | ||
| ../../test/ztest/unit/math/basic/arithmetic/test_find_equal_int16_ztest.c | ||
| ../../test/ztest/unit/math/basic/arithmetic/test_find_max_abs_int32_ztest.c | ||
| ../../test/ztest/unit/math/basic/arithmetic/test_find_min_int16_ztest.c | ||
| ../../test/ztest/unit/math/basic/arithmetic/test_gcd_ztest.c | ||
| ../../test/ztest/unit/math/basic/arithmetic/test_norm_int32_ztest.c | ||
| ) | ||
|
|
||
| zephyr_library_sources(${MATH_ZTEST_SOURCES}) | ||
|
|
||
| set_source_files_properties(${MATH_ZTEST_SOURCES} | ||
| PROPERTIES COMPILE_DEFINITIONS "CONFIG_NUMBERS_VECTOR_FIND=1;CONFIG_NUMBERS_NORM=1;UNIT_TEST=1" | ||
| ) | ||
| endif() |
There was a problem hiding this comment.
Either the commit message is wrong or there was some mistake here: "Make the maths tests available on native sim target.". These tests are currently built exclusively for native_sim.
There was a problem hiding this comment.
This is a good change but I would like to have the option to build native_sim in two variants. With Zephyr heap and with system heap. Please add a Kconfig enabling such a choice.
There was a problem hiding this comment.
fixed, have 2 options now.
Add native_sim board configuration and support in the build script. This allows building and running tests on the host using Zephyr's native_sim target. native_sim leverages the POSIX architecture, but the libfuzzer support specifically requires CONFIG_ARCH_POSIX_LIBFUZZER to be set. Therefore, this wraps fuzzer-specific code in ipc.c and the build of fuzz.c behind this config to allow clean compilation on the standard native_sim board. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add native_sim board target to the sof-qemu-run scripts, and add an option to additionally run it under valgrind. The default build directory is set to ../build-native_sim Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
When building the firmware for native_sim, debugging allocations with host machine tools like Valgrind is constrained due to Zephyr's internal minimal libc tracking the heap manually via static pools. By bypassing Zephyr's memory interception on native_sim using nsi_host_malloc, dynamically tracked memory can surface appropriately to Valgrind memory checkers without causing a libc heap pool panic. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
In file included from /home/lrg/work/sof2/sof/zephyr/sof_shell.c:14:
/home/lrg/work/sof2/sof/zephyr/sof_shell.c: In function 'cmd_sof_module_heap_usage':
/home/lrg/work/sof2/sof/zephyr/sof_shell.c:66:77: error: 'struct module_config' has no member named 'heap_bytes'
66 | icd->id, usage, hwm, comp_mod(icd->cd)->priv.cfg.heap_bytes);
| ^
/home/lrg/work/sof2/zephyr/include/zephyr/shell/shell.h:1292:47: note: in definition of macro 'shell_print'
1292 | shell_fprintf_normal(_sh, _ft "\n", ##__VA_ARGS__)
| ^~~~~~~~~~~
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Keep spinning in case user needs to inspect status via monitor. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
lgirdwood
force-pushed
the
native_sim
branch
2 times, most recently
from
4cb1db9 to
f2a31fa
Compare
When building the native_sim fuzzer, the host allocator does not possess the strict bounds of the internal Zephyr memory pools. If the fuzzer generates a malformed payload requesting an excessively large size (e.g. 4GB), it passes directly to the host ASAN allocator which aborts due to OOM or protection limits. Adding a 16MB cap allows these to fail gracefully. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
The fuzzer's payload size posix_fuzz_sz is provided by libFuzzer as a size_t. Declaring it as a uint8_t in the ipc test harness resulted in silent payload truncation (maximum 255 bytes) causing incomplete corpus generation. This corrects the types between fuzz.c and ipc.c. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add the math basic-arithmetic ztests (test_crc32_ztest.c, test_find_*_ztest.c, test_gcd_ztest.c, test_norm_int32_ztest.c) to the SOF boot-test build under zephyr/test/CMakeLists.txt when both CONFIG_SOF_BOOT_TEST_STANDALONE and CONFIG_ZTEST are enabled. The math helpers themselves (find_*_int*, norm_int32) are gated by CONFIG_NUMBERS_VECTOR_FIND / CONFIG_NUMBERS_NORM in src/math/numbers.c and are now enabled per-board in app/boards/native_sim.conf, where CONFIG_ZTEST is set. This keeps the gating clean and explicit per board rather than coupling math source compilation to CONFIG_ZTEST. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Run the boot tests and quit when done. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
The xtensa-build-zephyr.py script attempts to parse and copy the zephyr.ri (rimage) file for reproducible checksums and installation. Since the native_sim platform does not produce an rimage file, building it with this script results in a FileNotFoundError. Adding native_sim to the list of exceptions resolves the build failure. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
lyakh
left a comment
lyakh
left a comment
There was a problem hiding this comment.
one critical suspected breakage, should be checked at least
| print("\n[sof-qemu-run] No crash detected. (Skipping QEMU monitor interaction for native_sim)") | ||
| else: | ||
| print("\n[sof-qemu-run] Process is no longer alive, cannot extract registers.") | ||
| print("\n[sof-qemu-run] No crash detected. Interacting with QEMU Monitor to grab registers...") |
There was a problem hiding this comment.
can the second sentence be rephrased a bit like "Requesting registers from the QEMU Monitor?" I'm not sure I'd say "I'm going to interface with my neighbour to ask if they can borrow me 2 potatoes" :-)
| void *ptr; | ||
| void *raw; | ||
|
|
||
| if (bytes > 16 * 1024 * 1024) { |
There was a problem hiding this comment.
rmalloc_align() is also the entry point to DRAM (L3) allocations, where we already now have possibly more than 16M and we do large allocations there e.g. when downloading LLEXTs. I could imaging a very large LLEXT library with large "cold" parts that never get copied to SRAM. Can we make this something ridiculously large like 1G or maybe configurable? Not critical for now.
| sof_app_main(); | ||
| #if CONFIG_SOF_BOOT_TEST && defined(QEMU_BOOT_TESTS) | ||
| #if CONFIG_SOF_BOOT_TEST | ||
| sof_run_boot_tests(); |
There was a problem hiding this comment.
I think this breaks platforms where CONFIG_BOOT_TEST_ALLOWED=y (e.g. PTL) because on them the boot test should be run later, as it's currently done.
Add support for native sim target and include being able to run under valgrind. This should support all cmocka tests as ztests meaning more/all can be removed. Will be added to CI soon.